Automatic generation of Multiple Alternate Choice (MAC) test item stems by applying Causal Relation Explication, Addition and Manipulation (CREAM) to pre-processed source documents

A new method for generating the stem components of Multiple Alternate Choice (MAC) test items is proposed which makes use of a theory about causal coherence relation primitives. A trial of the Causal Relation Explication, Addition and Manipulation (CREAM) method is described which uses a domain specific evaluation method. The domain expert assigns a usability category to each MAC stem from a bank consisting of a total of 40 relevant MAC stems. 20 of the stems contained in the item bank were generated using the CREAM methodology while the other 20 MAC stems were created using a traditional approach. The results show 5 generated stems were assigned category A (use without changes) and a further 9 were assigned category B (use following minor changes). Further performance improvements are expected in future experiments by applying new pre-processing techniques to the source documents.